A storage server is a computer that provides access to information that is stored on one or more storage devices connected to the storage server, such as disk drives (“disks”), flash memories, or storage arrays. The storage server includes an operating system that may implement a storage abstraction layer such as a file system to logically organize the information as a hierarchical structure of storage objects such as directories and files on a storage device (e.g., disk). Each file may be implemented as set of data structures, e.g., data blocks, configured to store information, such as the actual data for the file.
The representation of the file system on disk may be block-based using, e.g., 4 kilobyte (kB) blocks, and using inodes to describe the files. An inode is a data structure which stores information about a file, directory, or other file system such as user and group ownership, access mode (read, write, execute permissions) and type of file. An inode for a file may include pointers to blocks on disk constituting the actual file.
A storage server may be configured to operate according to a client/server model of information delivery to allow one or more clients access to data stored on the storage server. Access may be provided by the storage server using a file-level service such as used in a network-attached storage (NAS) environment, a block-level service such as used in a storage area network (SAN) environment, a service providing both file-level and block-level access, a content-level service, or any other data access service implemented by the storage server. In this model, the client may comprise an application executing on a computer that “connects” to the storage server over a computer network, such as a point-to-point link, shared local area network, wide area network or virtual private network implemented over a public network, such as the Internet. A client may access the storage devices by submitting access requests to the storage server, for example, a “write” request to store client data included in a request to storage devices or a “read” request to retrieve client data stored in the storage devices.
Multiple storage servers may be networked or otherwise connected together as a network storage system to provide access to storage objects of the storage servers. In order to optimize the use of physical resources in a networked environment, data storage requests may be distributed among the storage servers for back-up purposes to protect against disasters with one of the storage servers or for avoiding processing bottlenecks at one of the storage servers. Accordingly, a storage object such as a file, for instance, may be copied from one storage server (referred to herein as the “source” or “source storage server”) to another storage server (referred to herein as the “destination” or “destination storage server”) for providing a copy of the file at the destination. When implemented to alleviate the processing load at the source, the copy operation may be referred to as “migrating” or “copy-offloading” the file from the source to the destination since the destination performs storage requests on the file to offload processing of the file by the source.
Copy offloading is particularly advantageous when the client connected to the storage server is a computer (host or primary client) implementing computer virtualization techniques for servicing requests from other computers (secondary clients) connected to the host. Virtualization is a process by which the underlying physical components of a computer are abstracted into logical components and may be presented as multiple virtual machines, giving the appearance to users of many computers which are operative on a single physical computer. By utilizing virtualization, the host is optimized to handle data requests from secondary clients by dedicating one virtual machine to one of the secondary clients, dedicating another virtual machine to another secondary client, etc.
In support of host virtualization capabilities, a storage server may maintain a type of storage object referred to as a “vdisk” to emulate a disk in a virtualized environment for each virtual machine. A vdisk may include data constituting operating system, application, configuration, and user files, as well as export controls and operation restrictions to mimic that of a physical disk. When a secondary client requests data from a virtual machine on the host, the host accesses a vdisk at the storage server which is associated with the particular virtual machine and performs the requested data retrieval or storage operation on the vdisk.
In the creation of vdisks, a storage server may use the configuration of a previously created vdisk (“existing vdisk”) so that the configuration of a new vdisk need not be manually constructed. Typically, vdisks constitute a portion of the same data (e.g., operating system and application files) as other vdisks, so blocks storing data may be commonly referenced by the vdisks to avoid storage of redundant blocks. To accomplish this, a storage server may copy the inode of an existing vdisk in a process referred to as “cloning” to allow the new vdisk to reference the same blocks as the existing vdisk. Any future changes to the new vdisk (e.g., configuration changes or new data stored on the vdisk) may then be written to new blocks followed by an update to the inode of the new vdisk. In this way, new vdisks are created to optimize storage capacity by avoiding redundant data blocks and to eliminate or reduce the need for manual construction of a new vdisk.
Advantageously, vdisks may be distributed between storage servers to further optimize performance of the storage servers by offloading the vdisk from a heavily loaded storage server to a less loaded storage server. When migrating vdisks, however, a decrease in performance of the storage system may occur. Network bandwidth for servicing normal data requests from primary clients may be diminished while data is migrated between storage servers. Further, since each vdisk may constitute a substantial amount of data, a lengthy transfer period delays the ability of the destination to judiciously service requests intended for the vdisk. This delay may even amount to weeks in certain cases.
A technique for overcoming these limitations includes migrating only those blocks of the vdisk which are not already available at the destination and may be referred to as “deduplication”. By avoiding the migration of duplicate data, a storage server may advantageously conserve processing resources for performing other storage tasks, as well as optimize overall storage capacity of the system. One conventional deduplication approach involves dividing the vdisk into fixed or variable portions at the source and generating a fingerprint for each portion. The fingerprint may be, for example, a checksum operation (checksum) of the underlying data and is operative as a unique identifier for such data but constitutes a smaller size than the underlying data. In lieu of sending the underlying data, the source sends only the fingerprint to the destination whereby a determination is made whether the fingerprint already exists at the destination. Only those portions of data for which fingerprints are not already at the destination are then sent to the destination. With certain checksum algorithms, however, a “collision” may occur where a fingerprint may not uniquely identify the underlying blocks. This occurs when a checksum for one portion of blocks results in the same checksum for another portion of blocks. An adverse consequence of a collision includes potentially the wrong blocks being sent to the destination.
An alternative approach for determining blocks already available at the destination involves identifying a prior version of a storage object at the destination and providing only the changed blocks between the current version and prior version to the destination. A version of a storage object involves a copy of blocks of a previous version to result in a duplicate set of blocks to which new data may be written without modifying data of the previous version. A version relationship thus generally indicates a portion of common data (e.g., duplicate data) as between two or more storage objects. With vdisks, however, the versioning approach may be counterproductive when implementing storage savings techniques since creating multiple versions of a vdisk involves the storage of redundant data at the source. Accordingly, while storage savings may be achieved at the destination, such savings are achieved at the expense of additional storage capacity required at the source. This alternative approach is therefore deficient in efficiently off-loading storage objects from a source to destination while optimizing overall storage system capacity.